On Sequential Estimation of the Renewal Function, Optimal

Stochastic approximation methodology is applied to the problem of estimating an optimal block replacement policy. Several improvements to minimize the asymptotic mean square error of the estimator are considered. A stopping time is then defined with the objective of a fixed-width confidence interval for the optimal policy. A nonparametric estimator is used for sequential estimation of the renewal function at a single point and over a bounded interval of real numbers. Several strongly consistent estimates of the asymptotic variance are given. Sequential fixed-width confidence interval methods are shown to be effective for estimating the renewal function at a point. A stopping time is defined for minimum (integrated) risk estimation over an interval. It is shown that bootstrap methods can be used to construct asymptotically correct confidence bands for the renewal function over an interval. Let R be a statistical function which depends on a distribution F and is defined on an interval of real numbers. Suppose R may be estimated using a random sample from F. Motivated by the problem of a fixed-width confidence band for R, a stopping time is defined that is analogous to those available for fixed-width confidence interval estimation of a real-valued parameter. The proposed stopping time can be used in conjunction with a bootstrap confidence band procedure. Sufficient conditions are given for the sequential confidence band to have the correct asymptotic coverage probability as the desired band width goes to zero. Several properties of the stopping time, such as asymptotic efficiency, are addressed with reference to bootstrap procedures. Stopping times for variable width confidence bands are also considered.